Dynamics of Structural Learning with an Adaptive Forgetting Rate

Structural learning with forgetting is a prominent method of multilayer, feedforward neural network complexity regularization. The level of regularization is controlled by a parameter known as the forgetting rate. The goal of this paper is to establish a dynamical system framework f o r the study of structural learning both to ofSer new insights into this methodology and to potentially provide a means of either developing new or analytically justijjying existing forgetting rate adaptation strategies. The resulting nonlinear model of structural learning is analyzed by developing a general linearized equation.for the case of a quadratic error,function. This analysis demonstrates the effectiveness of an adaptive forgetting rate. A simple example is provided to illustrate our approach.